Systems and methods for assisting a robotic apparatus

ABSTRACT

Systems and methods assisting a robotic apparatus are disclosed. In some exemplary implementations, a robot can encounter situations where the robot cannot proceed and/or does not know with a high degree of certainty it can proceed. Accordingly, the robot can determine that it has encountered an error and/or assist event. In some exemplary implementations, the robot can receive assistance from an operator and/or attempt to resolve the issue itself. In some cases, the robot can be configured to delay actions in order to allow resolution of the error and/or assist event.

COPYRIGHT

A portion of the disclosure of this patent document contains materialthat is subject to copyright protection. The copyright owner has noobjection to the facsimile reproduction by anyone of the patent documentor the patent disclosure, as it appears in the Patent and TrademarkOffice patent files or records, but otherwise reserves all copyrightrights whatsoever.

BACKGROUND Technological Field

The present application relates generally to robotics, and morespecifically to systems and methods for assisting a robotic apparatus.

Background

In some cases, robots can operate autonomously with little to no directoperator control. However, while these robots operate autonomously,these robots can run into situations where they can get stuck and/orhave errors and/or cannot proceed because of external causes (e.g.,blocked paths).

Many contemporary robots work in controlled environments with littleconnectivity and monitoring. Accordingly, operators often do not knowthe needs of the robots and the robots do not have robust procedures toask for help. In some cases, this can lead to the robots causing damageand/or other sub-optimal behavior of the robots. Moreover, in somecases, robots follow procedures for recovery that may even hamper theability of operators to assist them. Accordingly, there is a need forimproved systems and methods for assisting a robot.

SUMMARY

The foregoing needs are satisfied by the present disclosure, whichprovides for, inter alia, apparatus and methods for dynamic routeplanning in autonomous navigation. Example implementations describedherein have innovative features, no single one of which is indispensableor solely responsible for their desirable attributes. Without limitingthe scope of the claims, some of the advantageous features will now besummarized.

In a first aspect, a method for operating a robot is disclosed. In oneexemplary implementation, the method includes: causing the robot toperform a task in an environment; generating data about the environmentwith one or more sensors of the robot as the robot performs the task;determining from the data that the robot is at least partiallyobstructed from performing the task; displaying a first alert indicatingat least in part that the robot cannot continue the task after apredetermined first delay after the determination that the robot is atleast partially obstructed; attempting to cause the robot to perform thetask after a predetermined second delay after displaying the alert; anddisplaying a second alert indicating at least in part that the robot isat least partially obstructed after a predetermined third delay afterthe attempting to cause the robot to perform the task.

In one variant, the task comprises cleaning a floor. In another variant,the method comprises receiving operating instructions from an operatorand continuing the performance of the task based at least in part on theoperating instructions. In another variant, receiving operatinginstructions from the operator comprises receiving a determination thata button of the robot has been pushed, wherein the pushing of the buttonis an instruction to proceed performing the task notwithstanding theobstacle. In another variant, receiving operating instructions from theoperator comprises displaying paths of travel and receiving a selectionof the displayed paths.

In another variant, the method further comprises receiving data from anexternal sensor, and based at least in part on the data from theexternal sensor, determining if the robot is still at least partiallyobstructed from performing the task. In another variant, the methodfurther comprises identifying that the robot is at least partiallyobstructed through object recognition.

In a second aspect, a robot is disclosed. In one exemplaryimplementation, the robot includes: one or more sensors configured togenerate sensor data about an environment; one or more actuatorsconfigured to perform a first robotic task; a user interface configuredto display and receive information; and a processor configured to: causethe one or more actuators to perform the first robotic task; detect,based at least in part on the sensor data, an obstacle while performingthe first robotic task; actuate the one or more actuators so that therobot is not performing the first robotic task in response to, at leastin part, the detected obstacle; display on the user interface an assistview comprising an alert indicative at least in part of the detectedobstacle and one or more selectable positions of the robot; receivethrough the user interface a selected one of the one or more selectablepositions of the robot; and actuate the actuators based at least in parton a selected one of the one or more selectable positions of the robot,and continue performing a task substantially similar to the firstrobotic task.

In one variant, the robot further comprises a communications unitcommunicatively coupled to a remote network configured to monitor therobot. In another variant, the processor is further configured toidentify the obstacle. In another variant, the processor is furtherconfigured to learn to identify the obstacle through a machine learningprocess. In another variant, the robot comprises a floor cleaner. Inanother variant, the robotic task is navigation. In another variant, theselectable positions of the robot comprise trajectories of the robot.

In a third aspect, a method for assisting a floor cleaning robot isdisclosed. In one exemplary implementation, the method comprises:displaying an assist view graphically representing that the floorcleaning robot is obstructed from cleaning; receiving operatinginstructions through the assist view, the operating instructionscomprising displaying a plurality of paths of travel for the robot on auser interface, and receiving a selection of the displayed paths; andcausing the floor cleaning robot to clean in accordance to the operatinginstructions.

In one variant, displaying the assist view further comprises: receivingdata from a sensor of the robot; identifying an assist event based atleast in part on the received data; and generating a graphic indicatingthe identified assist event.

In another variant, the method further comprises transmitting the assistview to a remote network. In another variant, causing the floor cleaningrobot to clean comprises: causing the floor cleaning robot to navigatealong a predetermined path; and causing the floor cleaning robot toscrub the floor along the predetermined path.

In another variant, the method further comprises delaying causing thefloor cleaning robot to clean in accordance to the operatinginstructions at least a predetermined time from displaying the assistview.

In another variant, causing the floor cleaning robot to clean inaccordance to the operating instructions occurs after receiving a signalindicating that an assist button on the robot has been pushed.

In a fourth aspect, a non-transitory computer-readable storage apparatusis disclosed. In one exemplary implementation, the non-transitorycomputer-readable storage apparatus has a plurality of instructionsstored thereon, the instructions being executable by a processingapparatus to operate the robot. The instructions configured to, whenexecuted by the processing apparatus, cause the processing apparatus to:cause the one or more actuators to perform the robotic task; detect,based at least in part on the sensor data, an obstacle while performingthe robotic task; actuate the one or more actuators so that the robot isnot performing the robotic task in response to, at least in part, thedetected obstacle; display on the user interface an assist viewcomprising an alert indicative at least in part of the detected obstacleand one or more selectable positions of the robot; receive through theuser interface a selected one of the one or more selectable positions ofthe robot; and actuate the actuators based at least in part on aselected one of the one or more selectable positions of the robot, andcontinue performing the task.

In a fifth aspect, a system for assisting a robot is disclosed. In oneexemplary implementation, the system includes: a robot configured toperform tasks, the robot having one or more sensors to collect dataabout an environment of the robot, wherein the robot transmits an assistrequest when it encounters an obstacle in the environment detected inthe data of the one or more sensors; an external sensor configured tocollect data about at least a portion of the environment of the robot;and a remote network communicatively coupled to the robot and theexternal sensor, the remote network configured to: receive the assistrequest from the robot, obtain data from the external sensor in responseto the assist request; and determine how to assist the robot.

In one variant, determining how to assist the robot further comprisesdetermining whether the robot can proceed.

In a sixth aspect, a remote network is disclosed. In one exemplaryimplementation, the remote network includes: a communications unitconfigured to receive sensor data and a request for assistance from arobot; and a processor configured to: identify an event of interest inthe received sensor data; determine whether to acquire more informationabout the event; based at least in part on the determination of whetherto acquire more information, request data from an external sensor aboutthe event; and based at least in part on the data from the externalsensor, send an assist request to an access point to assist the robot.

In a seventh aspect, a sensor external to a robot is disclosed. In oneexemplary implementation, the sensor is configured to configured tocollect data about at least a portion of the environment of the robot;detect an assist event in the data; and communicate with the robot toresolve the assist event.

In one variant, the sensor is further configured to communicate with aremote network to resolve the assist event.

These and other objects, features, and characteristics of the presentdisclosure, as well as the methods of operation and functions of therelated elements of structure and the combination of parts and economiesof manufacture, will become more apparent upon consideration of thefollowing description and the appended claims with reference to theaccompanying drawings, all of which form a part of this specification,wherein like reference numerals designate corresponding parts in thevarious figures. It is to be expressly understood, however, that thedrawings are for the purpose of illustration and description only andare not intended as a definition of the limits of the disclosure. Asused in the specification and in the claims, the singular form of “a”,“an”, and “the” include plural referents unless the context clearlydictates otherwise.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosed aspects will hereinafter be described in conjunction withthe appended drawings, provided to illustrate and not to limit thedisclosed aspects, wherein like designations denote like elements.

FIG. 1 is a process flow diagram of an exemplary method for operation ofa robot in accordance with some implementations of this disclosure.

FIG. 2A is a top view diagram illustrating a robot footprint of a robotin accordance with some implementations of this disclosure.

FIG. 2B is a top view diagram of a robot navigating around an object tono avail in accordance with some implementations of this disclosure.

FIG. 3 is a functional block diagram of a robot in accordance with someprinciples of this disclosure.

FIG. 4 is a functional block diagram of a system, which includes a robotcommunicatively and/or operatively coupled to a remote network inaccordance with some implementations of this disclosure.

FIG. 5 is a process flow diagram of an exemplary method for issuing anassist request in accordance with some implementations of thisdisclosure.

FIG. 6A-6C are displays that show assist views in accordance with someimplementations of the present disclosure.

FIG. 7 is a side view of an external sensor for providing informationabout a robot in accordance with some implementations of the presentdisclosure.

FIG. 8 is a functional block diagram illustrating an assist process inaccordance with some implementations of the present disclosure.

FIG. 9 is a process flow diagram of an exemplary method for operation ofa robot in accordance with some implementations of this disclosure.

FIG. 10 is a process flow diagram of an exemplary method for operationof a robot in accordance with some implementations of this disclosure.

FIG. 11 is a process flow diagram of an exemplary method for assisting arobot in accordance with some implementations of this disclosure.

All Figures disclosed herein are © Copyright 2017 Brain Corporation. Allrights reserved.

DETAILED DESCRIPTION

Various aspects of the novel systems, apparatuses, and methods disclosedherein are described more fully hereinafter with reference to theaccompanying drawings. This disclosure can, however, be embodied in manydifferent forms and should not be construed as limited to any specificstructure or function presented throughout this disclosure. Rather,these aspects are provided so that this disclosure will be thorough andcomplete, and will fully convey the scope of the disclosure to thoseskilled in the art. Based on the teachings herein, one skilled in theart would appreciate that the scope of the disclosure is intended tocover any aspect of the novel systems, apparatuses, and methodsdisclosed herein, whether implemented independently of, or combinedwith, any other aspect of the disclosure. For example, an apparatus canbe implemented or a method can be practiced using any number of theaspects set forth herein. In addition, the scope of the disclosure isintended to cover such an apparatus or method that is practiced usingother structure, functionality, or structure and functionality inaddition to or other than the various aspects of the disclosure as setforth herein. It should be understood that any aspect disclosed hereincan be implemented by one or more elements of a claim.

Although particular aspects are described herein, many variations andpermutations of these aspects fall within the scope of the disclosure.Although some benefits and advantages of the preferred aspects arementioned, the scope of the disclosure is not intended to be limited toparticular benefits, uses, and/or objectives. The detailed descriptionand drawings are merely illustrative of the disclosure rather thanlimiting, the scope of the disclosure being defined by the appendedclaims and equivalents thereof.

The present disclosure provides for improved remote operating and/ormonitoring of a robot. As used herein, a robot can include mechanicaland/or virtual entities configured to carry out complex series ofactions automatically. In some cases, robots can be machines that areguided and/or instructed by computer programs and/or electroniccircuitry. In some cases, robots can include electro-mechanicalcomponents that are configured for navigation, where the robot can movefrom one location to another. Such navigating robots can includeautonomous and/or self-driving cars, floor cleaners, rovers, drones,planes, boats, carts, and the like.

In some cases, robots can include appliances, machines, and/or equipmentautomated to perform one or more tasks. For example, a module can beattached to the appliances, machines, and/or equipment to allow them tooperate autonomously. In some implementations, the module can include amotor that drives the autonomous motions of the appliances, machines,and/or equipment. In some cases, the module causes the appliances,machines, and/or equipment to operate based at least in part onspoofing, such as by sending control signals to pre-existingcontrollers, actuators, units, and/or components of the appliances,machines, and/or equipment. The module can include sensors and/orprocessors to receive and generate data. The module can also includeprocessors, actuators, and/or any of the components described herein toprocess the sensor data, send control signals, and/or otherwise controlpre-existing controllers, units, and/or components of the appliances,machines, and/or equipment. Such appliances, machines, and/or equipmentcan include forklifts, tugs, boats, planes, industrial equipment, floorcleaners, stocking machines, home appliances, etc.

As referred to herein, floor cleaners can include floor cleaners thatare manually controlled (e.g., driven or remote controlled) and/orautonomous (e.g., using little to no direct user control). For example,floor cleaners can include floor scrubbers that a janitor, custodian,and/or other person operates and/or robotic floor scrubbers thatautonomously navigate and/or clean an environment. Similarly, floorcleaners can also include vacuums, steamers, buffers, mops, polishers,sweepers, burnishers, and/or devices intended to clean a floor.

Detailed descriptions of the various implementations and variants of thesystem and methods of the disclosure are now provided. While manyexamples discussed herein may refer to robotic floor cleaners, it willbe appreciated that the described systems and methods contained hereinare applicable to any kind of robot. Myriad other exampleimplementations or uses for the technology described herein would bereadily envisaged by those having ordinary skill in the art, given thecontents of the present disclosure.

Advantageously, the systems and methods of this disclosure at least: (i)allow robots to safely operate in environments even in unforeseencircumstances; (ii) provide comfort (e.g., by humans and/or animals)with robots by allowing robots to exhibit expected behaviors; (iii)allow for smooth operation of robots without undesired stoppages; and(iv) provide efficient maneuvering of robots. Other advantages arereadily discernable by one having ordinary skill in the art given thecontents of the present disclosure.

For example, a robot can navigate an environment, collecting data and/orperforming actions. By way of illustration, the robot can be a floorcleaner that cleans a floor as it navigates an environment. The robotcan collect and/or generate data. For example, the robot can havesensors that sense the environment of the robot, as will be laterdescribed in this disclosure with reference to FIG. 3. These sensors canproduce data about the environment. The robot can also have sensors thatmonitor the performance of the robot, such as on/off time, speed,statuses, software version, operation (e.g., performingactions/operations, such as cleaning), maintenance, etc. These sensorscan produce data about the robot. The robot can also generate maprelated data (e.g., data related to and/or that can be used for mappingand/or localization) containing information about the robot'senvironment and/or route that allow the robot to autonomously navigatean environment.

The robot can encounter events that trigger issues with the robot. Insome cases, the issues can be errors, where the robot has aperformance-related issue in software and/or hardware. In some cases,the issues can be scenarios where the robot is configured to seekadditional input from an operator to be able to determine how the robotshould perform. In some cases, these issues can be detected by one ormore sensors of the robot. Advantageously, determining that the eventhas occurred and appropriately requesting an assist can improve operatorexperience and/or increase confidence in the operation of the robot.Moreover, appropriate systems and methods for assists can allowefficient operation of tasks designated to the robot.

FIG. 1 is a process flow diagram of an exemplary method 100 foroperation of a robot in accordance with some implementations of thisdisclosure. Block 102 includes performing a task. For example, suchtasks can be a function associated with the robot, such as navigating,cleaning, moving objects, transporting objects, exploring, mapping,constructing, imaging, diagnosing, and/or any activity desired for arobot. In some cases, the robot can be used by an operator to performsuch a task.

Block 104 includes detecting an error and/or assist event during theperformance of the task. As used herein assists can relate to situationsin which operator assistance of a robot is desirable. For example,operator assistance can relate to operator direction, commands,statuses, etc. In some cases, assists can relate to scenarios caused byerrors. For example, errors can include a number of occurrences of arobot that result in impairment of one or more features of the robot.

By way of illustration, fatal errors can include catastrophic failures,loss of connection, emergency shutdowns, destruction, and otheroccurrences that cause the robot to have little to no functionality. Insome cases, there can be relatively little that can be done by the robotonce the catastrophic failure has occurred. In some cases, an operatormay need to repair and/or replace some portions of the robot before therobot can function again.

As another illustration, errors can include internal errors, which canbe errors relating to the internal workings of the software or hardwareof the robot. For example, these internal errors can include softwarecomponent crashes, stack errors and/or overflows, memory gaps and/ormemory errors, software component hang-ups and/or freezes, etc. Asanother example, internal errors can include mechanical-electricalerrors and/or failures, such as those related to mechanical and/orelectrical systems of the robot, such as actuators, power system,controllers/computers, relays, connectors, cables, etc. Internal errorscan also include collisions and/or falsely identified collisions (e.g.,where a robot determines, through a mistake in software and/or hardware,that it has collided into something when it has not).

As another illustration, assists can relate to a failure of the robot toproceed with a task, such as a movement, operation, and/or other desiredactivity. For example, the robot can be inhibited from proceeding with aparticular activity. In some implementations, a robot can be configuredto perform a task, such as navigating, cleaning, moving objects,transporting objects, exploring, mapping, constructing, imaging,diagnosing, and/or any activity desired for a robot. In someimplementations, a robot can be inhibited from completing the task byexternal occurrences, such as by a blockage (e.g., presence of anobstacle), loss/damage/destruction of a task-related object (e.g., anobject the robot is supposed to transport), environmentaldangers/hazards for the robot, spills (e.g., caused by the robot or theenvironment), dangers for the environment and/or objects/people in theenvironment, and/or external occurrences that can prevent a robot fromperforming a task. In some implementations, a robot can be inhibitedfrom completing the task by internal occurrences, such as perceptions(e.g., based at least in part on data from sensors of the robot) that itcannot proceed with the task (e.g., based at least in part on thepredetermined limits of the robot, such as the an amount of clearance inthe robot's predefined footprint parameters of itself as described withreference to FIG. 2A), detection of ghost obstacles (e.g., the robotsees an obstacle that does not exist), noise (e.g., sensor noise,reflections, and/or other unwanted additions/modifications to a signal),delocalization, slippage (e.g., errors in odometry where the robot canmistakenly determine it is somewhere else, including somewhere in whichit can hit and/or come close to an object), obstacles detected insidethe robot's predefined footprint parameters of itself, etc.

In order to illustrate a robot's predefined footprint of itself, FIG. 2Ais a top view diagram illustrating robot footprint 202 of robot 200 inaccordance with some implementations of this disclosure. Robot 200 canhave a predefined body shape and dimensions. Robot 200 can include abody with a plurality of sides, such as front side 204A, right side204C, left side 204D, and back side 204B. Robot 200 can also have a topside and a bottom side. A person having ordinary skill in the art, giventhe contents of the present disclosure, would appreciate that robot 200can have other sides as well, corresponding to the surfaces of robot200, which can vary by shape (e.g., rectangular, pyramidal, humanoid, orany other designed shape). By way of illustration, front side 204A canbe positioned on the forward-facing side of robot 200, where theforward-facing side is forward in the direction of forward movement ofrobot 200. Back side 204B can be positioned on the backward-facing sideof robot 200, where the backward-facing side is the side facing insubstantially the opposite direction of the forward-facing side. Rightside 204C can be the right-hand side relative to front side 204A, andleft side 204D can be the left-hand side relative to front side 204A.

The actual body of robot 200 is illustrated. However, footprint 202 canbe the size of robot 200 configured in the software and/or hardware ofrobot 200 for determining how it can navigate and/or perform tasks. Byway of illustration, footprint 202 can extend (as illustrated) beyondfront side 204A, right side 204C, left side 204D, and/or back side 204B,creating a difference between what robot 200 perceives is the size ofrobot 200 in software and/or hardware (e.g., footprint 202) and what theactual size of robot 200 is in reality. This can allow clearance betweenrobot 200 and/or objects in an environment. Advantageously, footprint202 can allow a safety buffer for navigation. The size and/or shape offootprint 202 can be based at least in part on a number of factors,including desired clearance, complexity of environments, tolerance toassists, risk of collision (e.g., the occurrence of, the amount ofdamage of, etc.), and/or other factors. As illustrated, footprint 202 isrectangular; however, footprint 202 can take on any shape desirable,such as square, triangular, rectangular, parallelogram asymmetricshapes, curves, etc. Moreover, footprint 202 is illustrated from a topview, seen as two-dimensional. However, in some cases, footprint 202 canbe three dimensional, reflecting robot 200's perception of robot 200from a plurality of surfaces of robot 200.

As illustrated, in some implementations, footprint 202 can be largerthan the actual size of robot 200. A person having ordinary skill in theart, given the contents of the present disclosure, can appreciate thatthe amount of clearance from each side of footprint 202 relative torobot 200 can vary independently at different points, wherein footprint202 can be a different shape from robot 200, have asymmetric clearances,and/or variations throughout. In some implementations, footprint 202 canbe substantially similar in size and/or shape to robot 200, creating noclearance. Advantageously, having this substantial similarity can allowrobot 200 to closely navigate around objects and/or minimize assists.However, having this substantial similarity may provide increased risksof collisions due to noise and/or aberrations in mapping and/ornavigation of robot 200.

In some cases, robot 200 can fit through a space, yet footprint 202 cancause robot 200 to perceive that it cannot fit. In such a case, robot200 stopping because footprint 202 cannot transverse a space can giverise to an assist scenario. An operator could then determine if robot200 can actually traverse the space and instruct robot 200 accordingly.

As another illustration, assists can relate to a robot getting stuck.For example, a robot can get stuck if it has limitations in performingtasks. By way of illustration, a robot could go into an area from whichit cannot navigate out autonomously. Such can be the case where a robothas navigational limitations, including an inability to move in reverseand/or operate in certain direction(s) (e.g., left, right, front, back,etc.). In such cases, an operator may desire to give a robotinstructions to combine actions within its limitations to get out and/orphysically turn the robot. As another example, a robot can attempt toproceed past an object a plurality of times to no avail. A robot mayalso get stuck due to internal errors and/or other causes.

In some implementations of this disclosure, assistance can includeinstructions and/or interactions with the robot. For example, anoperator can respond to an error and/or assist event by moving the robot(e.g., taking control of the robot, touching the robot, picking up therobot, rotating the robot, etc.), telling the robot to proceed,resetting the robot, and/or other assistance.

A person with ordinary skill in the art, given the contents of thepresent disclosure, would appreciate that there are any number of casesin which assisting a robot can be desirable, and the systems and methodsof this disclosure are readily adaptable to any of those situations.

FIG. 2B is a top view diagram of robot 200 navigating around an object250 to no avail in accordance with some implementations of thisdisclosure. Robot 200 can initially travel along route 252A. An object250 can be in position 254A, located along route 252A. Robot can detectthe object using one or more sensors (e.g., of sensors unit 312 asdescribed with reference to FIG. 3) As a result, robot 200 can attemptto navigate around position 254A by travelling along route 252B, whichcan appear to be clear while robot 200 is planning. However, the object250 can move to position 254B when robot 200 attempts to travel alongroute 252B. Accordingly, robot 200 can be stymied from proceeding. Insome cases, robot 200 can again re-route to route 252A. However, theobject 250 can then return to position 254A before robot 200 canreroute. This re-routing can occur a plurality of times to no avail.Accordingly, robot 200 can request an assist after a predeterminednumber of re-route attempts. The predetermined number of re-routeattempts can be determined based at least in part on: the amount of timedesired for before robot 200 calls for an assist, the appearance ofrobot 200 if it is deliberating (e.g., it may be undesirable for robot200 to appear confused in front of the public, so fewer reroutes may bedesired before an assist is called), the desired number of assists(e.g., it may be desirable to have more assists for safety and/or otherreasons, or fewer assists to reduce robot dependency on operators), thefrequency of rerouting, empirical number of reroutes for resolving anissue, and/or other considerations.

Returning to FIG. 1, block 106 can include calling for an assist.Calling for an assist can include displaying indicative of an assist onthe robot, sending a message to an operator (e.g., via wireless or wiredsignal), communicating to a remote operating center, and/or sending anycommunication indicative at least in part of an assist. The assist canoccur by an operator physically assisting the robot (e.g., moving therobot, moving obstacles, moving objects related to a task of a robot,etc.) and/or issuing directions to the robot (e.g., through a userinterface, wireless signal, wired signal, etc.).

FIG. 3 is a functional block diagram of a robot 200 in accordance withsome principles of this disclosure. As illustrated in FIG. 3, robot 200can include controller 304, memory 302, user interface unit 318, mappingand localization unit 324, sensors unit 312, actuators unit 320, andcommunications unit 322, as well as other components and subcomponents(e.g., some of which may not be illustrated). Although a specificimplementation is illustrated in FIG. 3, it is appreciated that thearchitecture can be varied in certain implementations as would bereadily apparent to one of ordinary skill given the contents of thepresent disclosure. As used herein, robot 200 can be representative atleast in part of any robot described in this disclosure.

Controller 304 can control the various operations performed by robot200. Controller 304 can include one or more processors (e.g.,microprocessors) and other peripherals. As used herein, processor,microprocessor, and/or digital processor can include any type of digitalprocessing device such as, without limitation, digital signal processors(“DSPs”), reduced instruction set computers (“RISC”), general-purpose(“CISC”) processors, microprocessors, gate arrays (e.g., fieldprogrammable gate arrays (“FPGAs”)), programmable logic device (“PLDs”),reconfigurable computer fabrics (“RCFs”), array processors, securemicroprocessors, specialized processors (e.g., neuromorphic processors),and application-specific integrated circuits (“ASICs”). Such digitalprocessors can be contained on a single unitary integrated circuit die,or distributed across multiple components.

Controller 304 can be operatively and/or communicatively coupled tomemory 302. Memory 302 can include any type of integrated circuit orother storage device configured to store digital data including, withoutlimitation, read-only memory (“ROM”), random access memory (“RAM”),non-volatile random access memory (“NVRAM”), programmable read-onlymemory (“PROM”), electrically erasable programmable read-only memory(“EEPROM”), dynamic random-access memory (“DRAM”), Mobile DRAM,synchronous DRAM (“SDRAM”), double data rate SDRAM (“DDR/2 SDRAM”),extended data output (“EDO”) RAM, fast page mode RAM (“FPM”), reducedlatency DRAM (“RLDRAM”), static RAM (“SRAM”), flash memory (e.g.,NAND/NOR), memristor memory, pseudostatic RAM (“PSRAM”), etc. Memory 302can provide instructions and data to controller 304. For example, memory302 can be a non-transitory, computer-readable storage apparatus and/ormedium having a plurality of instructions stored thereon, theinstructions being executable by a processing apparatus (e.g.,controller 304) to operate robot 200. In some cases, the instructionscan be configured to, when executed by the processing apparatus, causethe processing apparatus to perform the various methods, features,and/or functionality described in this disclosure. Accordingly,controller 304 can perform logical and arithmetic operations based onprogram instructions stored within memory 302. In some cases, theinstructions and/or data of memory 302 can be stored in a combination ofhardware, some located locally within robot 200, and some located remotefrom robot 200 (e.g., in a cloud, server, network, etc.).

In some implementations, sensors unit 312 can comprise systems and/ormethods that can detect characteristics within and/or around robot 200.Sensors unit 312 can comprise a plurality and/or a combination ofsensors. Sensors unit 312 can include sensors that are internal to robot200 or external, and/or have components that are partially internaland/or partially external. In some cases, sensors unit 312 can includeone or more exteroceptive sensors, such as sonars, light detection andranging (“LIDAR”) sensors, radars, lasers, cameras (including videocameras (e.g., red-blue-green (“RBG”) cameras, infrared cameras, 3Dcameras, thermal cameras, etc.), time of flight (“TOF”) cameras,antennas, motion detectors, microphones, and/or any other sensor knownin the art. In some implementations, sensors unit 312 can collect rawmeasurements (e.g., currents, voltages, resistances, gate logic, etc.)and/or transformed measurements (e.g., distances, angles, detectedpoints in obstacles, etc.). In some cases, measurements can beaggregated and/or summarized. Sensors unit 312 can generate data basedat least in part on measurements. Such data can be stored in datastructures, such as matrices, arrays, queues, lists, arrays, stacks,bags, etc. In some implementations, the data structure of the sensordata can be called an image.

In some implementations, sensors unit 312 can include sensors that canmeasure internal characteristics of robot 200. For example, sensors unit312 can measure temperature, power levels, statuses, and/or any othercharacteristic of robot 200. In some cases, sensors unit 312 can beconfigured to determine location using the odometry of robot 200. Forexample, sensors unit 312 can include proprioceptive sensors, which cancomprise sensors such as accelerometers, inertial measurement units(“IMU”), odometers, gyroscopes, speedometers, cameras (e.g. using visualodometry), clock/timer, and the like. Odometry can facilitate autonomousnavigation and/or other autonomous actions of robot 200. This odometrycan include measurements of robot 200's position (e.g., where positioncan include robot's location, displacement and/or orientation, and cansometimes be interchangeable with the term pose as used herein) relativeto the initial location. Such data can be stored in data structures,such as matrices, arrays, queues, lists, arrays, stacks, bags, etc. Insome implementations, the data structure of the sensor data can becalled an image.

Mapping and localization unit 324 can receive sensor data from sensorsunit 312 to localize robot 200 in a map. In some implementations,mapping and localization unit 324 can include localization systems andmethods that allow robot 200 to localize itself in the coordinates of amap and/or relative to a location (e.g., an initialization location, endlocation, beacon, reference point, etc.). Mapping and localization unit324 can also process measurements taken by robot 200, such as bygenerating a graph (e.g., position graph) and/or map. In someimplementations, mapping and localization unit 324 can be at least aportion of controller 304.

In some implementations, user interface unit 318 can be configured toenable a user to interact with robot 200. For example, user interfaceunit 318 can include touch panels, buttons, keypads/keyboards, ports(e.g., universal serial bus (“USB”), digital visual interface (“DVI”),Display Port, E-Sata, Firewire, PS/2, Serial, VGA, SCSI, audioport,high-definition multimedia interface (“HDMI”), personal computer memorycard international association (“PCMCIA”) ports, memory card ports(e.g., secure digital (“SD”) and miniSD), and/or ports forcomputer-readable medium), mice, rollerballs, consoles, vibrators, audiotransducers, and/or any interface for a user to input and/or receivedata and/or commands, whether coupled wirelessly or through wires. Userinterface unit 318 can include a display, such as, without limitation,liquid crystal display (“LCDs”), light-emitting diode (“LED”) displays,LED LCD displays, in-plane-switching (“IPS”) displays, cathode raytubes, plasma displays, high definition (“HD”) panels, 4K displays,retina displays, organic LED displays, touchscreens, surfaces, canvases,and/or any displays, televisions, monitors, panels, and/or devices knownin the art for visual presentation. In some implementations userinterface unit 318 can be positioned on the body of robot 200. In someimplementations, user interface unit 318 can be positioned away from thebody of robot 200, but can be communicatively coupled to robot 200(e.g., via communication units including transmitters, receivers, and/ortransceivers) directly or indirectly (e.g., through a network, server,and/or a cloud).

In some implementations, communications unit 322 can include one or morereceivers, transmitters, and/or transceivers. Communications unit 322can be configured to send/receive a transmission protocol, such as,without limitation, one or more of BLUETOOTH®, ZIGBEE®, Wi-Fi, inductionwireless data transmission, radio frequencies, radio transmission,radio-frequency identification (“RFID”), near-field communication(“NFC”), infrared, network interfaces, cellular technologies such as 3G(3GPP/3GPP2), high-speed downlink packet access (“HSDPA”), high-speeduplink packet access (“HSUPA”), time division multiple access (“TDMA”),code division multiple access (“CDMA”) (e.g., IS-95A, wideband codedivision multiple access (“WCDMA”), etc.), frequency hopping spreadspectrum (“FHSS”), direct sequence spread spectrum (“DSSS”), globalsystem for mobile communication (“GSM”), Personal Area Network (“PAN”)(e.g., PAN/802.15), worldwide interoperability for microwave access(“WiMAX”), 802.20, long term evolution (“LTE”) (e.g., LTE/LTE-A), timedivision LTE (“TD-LTE”), global system for mobile communication (“GSM”),narrowband/frequency-division multiple access (“FDMA”), orthogonalfrequency-division multiplexing (“OFDM”), analog cellular, cellulardigital packet data (“CDPD”), satellite systems, millimeter wave ormicrowave systems, acoustic, infrared (e.g., infrared data association(“IrDA”)), and/or any other form of wireless data transmission.

As used herein, network interfaces can include any signal, data, orsoftware interface with a component, network, or process including,without limitation, those of the FireWire (e.g., FW400, FW800, FWS800T,FWS1600, FWS3200, etc.), universal serial bus (“USB”) (e.g., USB 1.X,USB 2.0, USB 3.0, USB Type-C, etc.), Ethernet (e.g., 10/100, 10/100/1000(Gigabit Ethernet), 10-Gig-E, etc.), multimedia over coax alliancetechnology (“MoCA”), Coaxsys (e.g., TVNET™), radio frequency tuner(e.g., in-band or OOB, cable modem, etc.), Wi-Fi (802.11), WiMAX (e.g.,WiMAX (802.16)), PAN (e.g., PAN/802.15), cellular (e.g., 3G,LTE/LTE-A/TD-LTE/TD-LTE, GSM, etc.), IrDA families, etc. As used herein,Wi-Fi can include one or more of IEEE-Std. 802.11, variants of IEEE-Std.802.11, standards related to IEEE-Std. 802.11 (e.g., 802.11a/b/g/n/ac/ad/af/ah/ai/aj/aq/ax/ay), and/or other wireless standards.

Communications unit 322 can also be configured to send/receive utilizinga transmission protocol over wired connections, such as any cable thathas a signal line and ground. For example, such cables can includeEthernet cables, coaxial cables, Universal Serial Bus (“USB”), FireWire,and/or any connection known in the art. Such protocols can be used bycommunications unit 322 to communicate to external systems, such ascomputers, smart phones, tablets, data capture systems, mobiletelecommunications networks, clouds, servers, or the like.Communications unit 322 can be configured to send and receive signalscomprising of numbers, letters, alphanumeric characters, and/or symbols.In some cases, signals can be encrypted, using algorithms such as128-bit or 256-bit keys and/or other encryption algorithms complyingwith standards such as the Advanced Encryption Standard (“AES”), RSA,Data Encryption Standard (“DES”), Triple DES, and the like.Communications unit 322 can be configured to send and receive statuses,commands, and other data/information. For example, communications unit322 can communicate with a user operator to allow the user to controlrobot 200. Communications unit 322 can communicate with a server/network(e.g., a remote network) in order to allow robot 200 to send data,statuses, commands, and other communications to the server. The servercan also be communicatively coupled to computer(s) and/or device(s) thatcan be used to monitor and/or control robot 200 remotely. Communicationsunit 322 can also receive updates (e.g., firmware or data updates),data, statuses, commands, and other communications from a server forrobot 200.

Actuators unit 320 can include any system used for actuating, in somecases to perform tasks. For example, actuators unit 320 can includedriven magnet systems, motors/engines (e.g., electric motors, combustionengines, steam engines, and/or any type of motor/engine known in theart), solenoid/ratchet system, piezoelectric system (e.g., an inchwormmotor), magnetostrictive elements, gesticulation, and/or any actuatorknown in the art. In some implementations, actuators unit 320 caninclude systems that allow movement of robot 200, such as motorizepropulsion. For example, motorized propulsion can move robot 200 in aforward or backward direction, and/or be used at least in part inturning robot 200 (e.g., left, right, and/or any other direction). Byway of illustration, actuators unit 320 can control if robot 200 ismoving or is stopped and/or allow robot 200 to navigate from onelocation to another location. Actuators unit 320 can also be configuredto actuate other instruments of robot 200, such as turning on/off water,turning on/off brushes, spraying water, turning on/off vacuums, movingvacuum hose positions, moving appendages (e.g., robotic arms, legs,hands, feet, etc.), and/or any other action.

One or more of the units described with respect to FIG. 3 (includingmemory 302, controller 304, sensors unit 312, user interface unit 318,actuators unit 320, communications unit 322, mapping and localizationunit 324, and/or other units) can be integrated onto robot 200, such asin an integrated system. However, in some implementations, one or moreof these units can be part of an attachable module. This module can beattached to an existing apparatus to automate so that it behaves as arobot. Accordingly, the features described in this disclosure withreference to robot 200 can be instantiated in a module that can beattached to an existing apparatus and/or integrated onto robot 200 in anintegrated system. Moreover, in some cases, a person having ordinaryskill in the art, given the contents of the present disclosure, wouldappreciate that at least a portion of the features described in thisdisclosure can also be run remotely, such as in a cloud, network, and/orserver. In some cases, controller 304 can perform include one or more ofthe units illustrated in FIG. 3, and/or other actions performed by robot200 can be processed by controller 304.

In some implementations, robot 200 can be communicatively coupled to aremote network. FIG. 4 is a functional block diagram of system 400,which includes robot 200 communicatively and/or operatively coupled toremote network 402 in accordance with some implementations of thisdisclosure. Remote network 402 can comprise a collection of hardware,software, services, and/or resources that can be invoked to instantiatea virtual machine, process, or other resource for a limited or definedduration, or an unlimited or undefined duration. Remote network 402 canbe communicatively and/or operatively coupled to a plurality of devices,systems, and/or servers, including devices and/or servers that haveaccess to the internet. One or more of access points, such as accesspoints 404A and 404B, can be devices, systems, and/or servers,including, but not limited to, computers, mobile devices, tablets, smartphones, cells phones, personal digital assistants, phablets, e-readers,smart watches, set-top boxes, internet streaming devices, gamingconsoles, smart appliances, and/or any device with access to theinternet and/or any network protocol. Although two access points areillustrated, there can be more or fewer access points as desired.

As used herein, remote network 402 can be operated: remote network 402can have onboard computers that can receive, process, and/or sendinformation. These computers can operate autonomously and/or undercontrol by one or more human operators. Similarly, remote network 402can have access points (e.g., access points 404A and 404B), which cansimilarly be used to operate remote network 402. The access points canhave computers and/or human operators that can receive, process, and/orsend information. Accordingly, references herein to operation of remotenetwork 402 can be applied to a human operator and/or a computeroperator.

In some implementations, one or more robots that are substantiallysimilar to robot 200 can be communicatively and/or operatively coupledto remote network 402. Each of these robots can communicates statuses,commands, and/or operative data to remote network 402. Remote network402 can also store and/or communicate statuses, commands, and/oroperative data to these one or more of robots.

A person having ordinary skill in the art, given the contents of thepresent disclosure, would appreciate that some portions of thisdisclosure may be performed by robot 200, network 402, and/or accesspoints 404A and/or 404B. Though certain examples may be described withreference to one or more of robot 200, network 402, and/or access points404A and/or 404B, it would be appreciated that the features of theexamples can be distributed amongst robot 200, network 402, and/oraccess points 404A and/or 404B to achieve substantially similar results.

FIG. 5 is a process flow diagram of an exemplary method 500 for issuingan assist request in accordance with some implementations of thisdisclosure. Block 502 includes detecting an error and/or assist event.The error and/or assist event can be detected using one or more sensorsof sensors unit 312. For example, exteroceptive sensors of sensor units312 can detect errors and/or assist events relating to the environmentof robot 200. By way of illustration, an error and/or assist event canrelate to a failure of robot 200 to proceed with a task. In some cases,robot 200 can use a sensor of sensors unit 312 (e.g., LIDAR, sonar,camera, etc.) to detect an object (e.g., an item, obstacle, person, flatobject, big object, glass, etc.).

For example, some sensors of sensors unit 312 emit and receive energy inorder to detect characteristics of an environment of robot 200. Suchenergy can include one or more of sound, light, electromagnetic waves,etc. By way of illustration, the sensor can be a LIDAR, which can detectcharacteristics of an environment by emitting light and measuring lightat a receiver (e.g., including reflected light). The presence of anobstacle can be detected by using the LIDAR. In some implementations,mapping and localization unit 324 can determine that the obstacle fallsalong the global or local path of robot 200. Accordingly, robot 200 candetermine that the path is impeded. Other examples include sound wavesused by a sonar. Various cameras also may emit and receive light, and/oruse filters to examine certain frequencies of light.

In some cases, an object and/or obstacle can be identified by robot 200,such as by detecting characteristics of the object and/or matching thedata from the sensor of sensors unit 312 to identified objects in alibrary of sensor data. In some cases, robot 200 may not specificallyidentify an object, but rather just detect the presence of an object. Asanother example, proprioceptive sensors of sensor units 312 can detecterrors and/or assist events relating to robot 200. The proprioceptivesensors can detect characteristics of robot 200 in which robot 200 maydesire assistance. In some implementations, once robot 200 has detectedan error and/or assist event, robot 200 can identify the error and/orassist event. Other errors and assist events can be substantiallysimilar to those described with reference to block 104 in method 100 ofFIG. 1.

In some cases, memory 302 can store a library of sensor data. In somecases, the sensor data can be associated at least in part with assistevents and/or errors. In some implementations, this library can includesensor data related to assist events and/or errors in differentconditions, such as sensor data related to objects with differentcompositions (e.g., materials, reflective properties, molecular makeup,etc.), different lighting conditions, angles, sizes, distances, clarity(e.g., blurred, obstructed/occluded, partially off frame, etc.), colors,surroundings, and/or other conditions. The sensor data in the librarycan be taken by a sensor (e.g., a sensor of sensors unit 312 or anyother sensor) and/or generated automatically, such as with a computerprogram that is configured to generate/simulate (e.g., in a virtualworld) library sensor data (e.g., which can generate/simulate theselibrary data entirely digitally and/or beginning from actual sensordata) from different lighting conditions, angles, sizes, distances,clarity (e.g., blurred, obstructed/occluded, partially off frame, etc.),colors, surroundings, and/or other conditions. The number of images inthe library can depend at least in part on one or more of the amount ofavailable data, the variability of the surrounding environment in whichrobot 200 operates, the complexity of objects, the variability inappearance of objects, physical properties of robots, thecharacteristics of the sensors, and/or the amount of available storagespace (e.g., in the library, memory 302, and/or local or remotestorage). In some implementations, at least a portion of the library maybe stored on a network (e.g., cloud, server, distributed network, etc.)and/or may not be stored completely within memory 302. As yet anotherexample, various robots (e.g., that are commonly associated, such asrobots by a common manufacturer, user, network, etc.) can be networkedso that data captured by individual robots are collectively shared withother robots. In such a fashion, these robots can be configured to learnand/or share sensor data in order to facilitate the ability to readilydetect and/or identify errors and/or assist events.

In some implementations, due to predetermined patterns in the sensordata and/or patterns learned by robot 200, robot 200 can identify anerror and/or assist event that has occurred. For example, errors and/orassist events can include categories of errors, such as fatal error,internal error, failure of the robot to proceed with a task, a robotgetting stuck, etc. Other identifications of categories of errors canalso be predetermined, including specific ones such as out of battery,obstacle in way, out of cleaning fluid, stack error, overheating, persondetected, and/or any predetermined identification of an error. In somecases, robot 200 can be configured to identify and/or recognize people,animals, objects, and/or other bodies detected in its sensors. A personhaving ordinary skill in the art, given the contents of the presentdisclosure, would appreciate that errors and/or assist events can begiven predetermined identifications as desired. For example, the type ofassistance desired can be different based on whether robot 200 detectspeople, animals, objects, and/or other bodies. Robot 200 can followlogical operations based on what robot 200 detects/identifies. A personand/or animal can move away from a trajectory of robot 200 in performinga task. Accordingly, in some cases, where robot 200 detects a personand/or animal, robot 200 can wait for the person and/or animal to moveaway. In some cases, robot 200 may not issue an assist event and/orerror at all and/or issue an assist event and/or error after a longerwait (e.g., predetermined time) in order to give time for the personand/or animal to move away. As another example, if robot 200 identifiesa box and/or static object in its path, robot 200 may issue an assistevent and/or error sooner (e.g., predetermined time) because the boxand/or static will probably not move away.

By way of illustration, robot 200 can detect fatal errors and/orinternal errors using certain sensors of sensors unit 312. For example,electrical switches, light sensors, and/or other sensors known in theart can be used to measure when fluids fall below desired levels inrobot 200. Robot 200 can monitor various components of robot 200,including implementing test procedures. Accordingly, robot 200 candetect when errors occur.

As another illustration, robot 200 can detect other errors, such asfailure of robot 200 to proceed with a task. In some cases, robot 200can detect obstacles with exteroceptive sensors, such as LIDAR, sonar,cameras (e.g., TOF cameras, RBG cameras, etc.), and/or any other sensorsconfigured to detect objects. In some cases, objects can be detectedthrough local planning, such as where robot 200 can detect objects byone or more points appearing locally in the data of the exteroceptivesensors. This local planning can be used by robot to navigate spaceswithin range of the exteroceptive sensors, such as by detecting thepresence of objects within the trajectory of robot 200. In this way,robot 200 can avoid crashing into objects. In some cases, robot 200 candetect objects through its global planning, wherein robot 200 can detectobjects from global maps. As another illustration, robot 200 can detectbeing stuck from the status of robot 200 (e.g., status generated basedat least in part from data from proprioceptive sensors).

In some implementations, detection of an error and/or assist event canbe based at least in part on a window of time. By way of illustration,an object can move in front of robot 200 and then later move away. Insome cases, if an error and/or assist event is based on a singlemeasurement, robot 200 can detect the object when a pixel in the sensordata indicates the presence of an object. However, this might not bedesirable. It can be desirable for robot 200 to have some robustness tonot stop and generate an alert about an error and/or assist eventwhenever it encounters an object. Moreover, relying on a singlemeasurement of a sensor to detect an error and/or assist event can beoverly influenced by noise. For example, due to ephemeral and/ortransient variations in measurements, false positives can occur (e.g.,objects detected, fluids perceived as low, etc.). Accordingly, detectionof an error and/or assist event can in some cases rely at least in parton a sustained and/or pattern of measurements from sensors in a windowof time.

The length of the window of time can be predetermined as desired. Insome cases, the length can be based at least in part on tolerance tofalse positives, dangers of failures to detect, empirical data,environmental features, and/or other considerations. In some cases, thelength of time can be approximately 15 seconds in order to give enoughtime for errors and/or assist events to be resolved without operatorassistance.

In some implementations, robot 200 can be stopped before robot 200detects an error and/or assist event. By way of illustration, in somecases, if robot 200 detects an obstacle (e.g., while it is actuating,such as by moving an arm or navigating), robot 200 can stop. Robot 200can then implement other courses of action to see if an error and/orassist event has occurred. For example, robot 200 can determine if thereis a route around the obstacle. If there is not another course of actionthat robot 200 determines robot 200 can take, robot 200 can then stay ina stopped position. In some cases, the obstacle can move away and therobot can proceed. In other cases, the obstacle can persist, and robot200 can detect an error and/or assist event.

In some cases, robot 200 being stopped includes stopping performance ofa task. For example, robot 200 can have actuators that perform a task,such as actuators configured to move brushes, vacuums, limbs, fluidoutput, wheels, etc. These actuations can be harmful to the environmentof robot 200 and/or wear robot 200 in certain scenarios. By way ofillustration, where robot 200 is a floor cleaner, robot 200 can have abrush that brushes a floor. If robot 200 is not moving, the brush mayrun on substantially the same place for a period of time; this runningcan lead to the brush causing damage to the floor. Accordingly, stoppingcan include activating an actuator to lift the brush deck and/orstopping the brush from revolving. As a further illustration in thissame example, the floor cleaner may also put water on the floor to beable to clean. If robot 200 is not moving, a vacuum in the back may notpick up all the water, causing spillage. In some implementations of thisdisclosure, robot 200 can shut off a valve and/or otherwise preventrobot 200 from putting water on the floor. As a further illustration inthis same example, a vacuum of the floor cleaner may wear if it is leftin the same place for an extended period. In some implementations ofthis disclosure, robot 200 can shut off the vacuum. As another example,robot 200 may have a motor running for motion. Leaving the motor onwhile robot 200 is not performing a task can lead to motor wear,pollution, and/or other harms.

In some implementations, a detected error and/or assist event in onecase may not be a detected error and/or assist event in others. Forexample, robot 200 can learn and/or observe how an operator handles anerror and/or assist event in one iteration, and perform substantiallysimilar actions if the same error and/or assist event occurs later. Byway of illustration, in a first iteration (e.g., of a plurality ofiterations), robot 200 can perform method 500. Robot 200 can associatethe actions taken by an operator with the circumstances that gave riseto the detected error and/or assist event in block 502. In a subsequentiteration, when robot 200 detects substantially similar circumstances,robot 200 can determine that it should take the actions associated withsuch circumstances, thereby avoiding performing at least a portion ofthe blocks of method 500. In some cases, robot 200 can learn toassociate the actions taken by the operator over a plurality ofiterations, such as through an iterative machine learning process.

Block 504 includes generating an alert about the error and/or assistevent. The alert can appear on a user interface unit 318, remote network402, and/or access points 404A and/or 404B. The alert can include ageneral statement that there has been an error and/or assist event,and/or the alert can include the identification of the error and/orassist event.

In some implementations, there can be logical operations performed todetermine where the assist appears. In some implementations, suchlogical operations can be performed by robot 200, such as by usingcontroller 304. The logical operations can determine where the assistappears based at least in part on the circumstances, such as the sensordata of robot 200, environmental conditions, identified objects in theenvironment, identified error and/or assist event, and/or any otherindicator of the circumstances of the error and/or assist event. By wayof illustration, where the assist event is a pathway being too narrow(and it is unclear if robot 200 can pass) robot 200 can issue an assistto remote network 402. Remote network 402 (and/or an operator of remotenetwork 402) can then determine if robot 200 can pass through and/ortell robot 200 to proceed. If robot 200 cannot pass through, at least inpart based on a determination of remote network 402, remote network 402can issue an assist request to a operator (e.g., who can see the assistrequest on user interface unit 318 and/or access points 404A and/or404B). In this way, the operator can go assist robot 200 (e.g., bymoving objects and/or robot 200). As another example, a blocked path maynot need a determination by remote network 402 (and/or an operator ofremote network 402), and swifter assistance could occur by going morequickly to a physical operator who could move robot 200 and/or theobjects. According, robot 200 can pass the assist to user interface 318and/or access points 404A and/or 404B (e.g., directly between robot 200and/or access points 404A and/or 404B, and/or through network 402)instead of waiting for remote network 402 to assist.

In some implementations, the logical operations can determine where tosend an assist contemporaneously. For example, robot 200 can issue anassist to a plurality of remote network 402, user interface unit 318,access points 404A and/or 404B, and/or other devices, where the assistis issued at substantially the same time. Similarly, in such cases, theassists may not have a hierarchy, wherein assistance from any device istreated similarly.

In some implementations, assists to remote network 402, user interfaceunit 318, access points 404A and/or 404B, and/or other devices can havea hierarchy, such as a hierarchy characterized at least in part on theauthority of the response and/or on the order in which assistance isrequested.

For example, a hierarchy based on authority can be implemented. In sucha case, robot 200 can place heavier weight to input by one device overother devices. For example, user interface unit 318 is located on robot200. Accordingly, assistance entered through user interface unit 318 canbe perceived as more apropos to the circumstances at hand. Accordingly,in some implementations, instructions from user interface unit 318 canoverride instructions received from others.

As another example, in some cases, remote network 402 can becharacterized at least in part as a centralized control center.Accordingly, instructions by remote network 402 (and/or operators ofremote network 402) can reflect at least in part experiences across aplurality of robots. In some cases, such instructions can overrideinstructions received from others. In some cases, one or more of accesspoints 404A and/or 404B, and/or other devices can be designated thehighest (e.g., most authoritative) in the hierarchy. For example, accesspoint 404A can be operated by a developer, manufacturer, etc. who canhave specialized knowledge of robot 200. Advantageously, having ahierarchy based on authority can allow robot 200 to sort throughdifferent instructions and/or perform more optimal actions.

As another example, in some implementations, the hierarchy can becharacterized by the order in which assistance is requested. Forexample, in some cases, an assist can be issued to remote network 402first relative to other devices. Advantageously, this can allow anenhanced user experience by resolving at least some issues withoutdisturbing an operator. If remote network 402 does not respond, and/oris unable to give appropriate instructions and/or assistance (e.g., aphysical interaction is desired), the assist can go to user interfaceunit 318, access points 404A and/or 404B, and/or other devices. Asanother example, assists can be issued to user interface unit 318 firstin order to allow a person with more circumstantial knowledge (e.g., aperson who is on-site) to deal with the issue first. In some cases,safety and/or other considerations can make immediate on-site assistancedesirable. Advantageously, having a hierarchy characterized by the orderin which assistance is requested can allow for the efficient resolutionof assists without overburdening resources. In some cases, there can bea predetermined delay between when each assist is issued. Suchpredetermined delay can be determined based at least in part on theamount of time to handle an assist, the response time of the device(e.g., a person and/or computer operating user interface unit 318,remote network 402, access points 404A and/or 404B, and/or otherdevices), empirical evidence of response times, workflow optimization,and/or other considerations.

In some implementations, the alert about the error and/or assist eventcan be in text form. By way of illustration, the alert can be a textmessage sent to an operator of robot 200 to inform the operator of theerror and/or assist event. The alert can be sent via SMS, the internet,radio, light, electric field, and/or any other form of communication(e.g., using communications unit 322). Advantageously, because someoperators of robot 200 may perform other tasks and/or manage otherrobots, sending the alert to the operator can allow the operator to beaway from robot 200 and/or performing other tasks.

By way of illustration, FIG. 6A is a display 600 that shows that anassist is needed in accordance with some implementations of the presentdisclosure. Display 600 can be displayed on user interface unit 318,remote network 402, and/or access points 404A and/or 404B. In someimplementations, display 600 can be on a stand-alone device (e.g., ofaccess points 404A and/or 404B) communicatively coupled to robot 200. Insome cases, the stand-alone device can be called an assistance device asit is used to assist robot 200.

Display 600 can have panel 602, which displays that an assist is needed.Panel 602 can also be used to display other text, such as “errordetected,” “assist requested,” and/or other text indicative at least inpart of an error and/or assist event.

Panel 614 can display further clarification of why there is an errorand/or assist event. For example, as illustrated in FIG. 6A, panel 614indicates that robot 200 is off path and that the path has an obstacle.A person having ordinary skill in the art, given the contents of thepresent disclosure, would appreciate that panel 614 can display any textindicative at least in part of the error and/or assist event that hasoccurred. For example, panel 614 can display information indicative atleast in part of fatal errors, internal errors, failure of the robot toproceed with tasks, a robot getting stuck, and/or any other event thatcan give rise to an error and/or assist event.

Panel 612 can display a direction for an operator and/or any other user.In some cases, the direction can be contextual based at least in part onthe error and/or assist event. For example, where robot 200 (e.g., whichcan be displayed in display 600 as robot icon 608) needs an assist todrive past an obstacle, panel 612 can display an instruction for anoperator and/or other user to deal with the obstacle. For example, asillustrated in FIG. 6A, panel 612 can state “remove or drive me past theobstacle, then push the BLUE button in the back.” In this example, theblue button in the back of robot 200 can be an external button that auser pushes to give robot 200 feedback, such as to indicate to robot 200that the operator and/or other user has taken care of the error and/orassist event. In some cases, the button can be a physical button that auser presses. The button can be any color as desired. The button can bea digital button and/or other input mechanism to robot 200, such asinput through user interface unit 318. In some implementations thebutton can be called an assist button as it is at least a portion ofassisting robot 200. In some implementations, even if robot 200 receivesinstructions through other avenues (e.g., display 600), robot 200 willnot proceed until the button has been pushed (and/or robot 200, remotenetwork 402, access point 404A and/or 404B receive a signal indicativeat least in part that the button has been pushed). In someimplementations, the button can cause robot 200 to re-localize anddetect where it is to continue operation. For example, if robot 200 hasbeen moved, robot 200 can determine where it is and what path it shouldtake using mapping and localization unit 324. In some implementations,there may be no button, and robot 200 can determine that the operatorand/or other user has taken care of the error and/or assist event fromdata from sensors unit 312. Similarly, once robot 200 has determinedthat the error and/or assist event has been resolved, robot 200 candetermine where it is and what path it should take using mapping andlocalization unit 324. A person having ordinary skill the art, given thecontents of the present disclosure, would appreciate that theinstructions in panel 612 can include any instruction given to addressthe error and/or assist event. As additional examples, panel 612 caninstruct an operator and/or user to fill fluids, adjust portions ofrobot 200 (e.g., propellers, brushes, wheels, etc.), move robot 200,move aspects of the environment of robot 200, turn/of robot 200, and/orany other instruction.

Panel 604 can include further information for an operator and/or otheruser regarding the error and/or assist event. For example, panel 604 caninclude a visual depiction of the error and/or assist event. Asillustrated in FIG. 6A, robot 200 is off path and/or is impeded by anobstacle. Robot icon 608 is, accordingly, displayed off path 610. Insome implementations, robot icon 608 can be positioned relative to othergraphics in panel 604 to indicate at least in part the position and/orpose of robot 200. In some cases, robot icon 608 can be off path 610 toreflect that robot 200 is off the path illustrated by path 610. In somecases, robot icon 608 can be on path 610 to reflect that robot 200 is onthe path illustrated by path 610.

In some implementations, panel 604 can include sensor data, such as datafrom one or more sensors of sensors unit 312 and/or external sensors.Advantageously, sensor data can give an operator context. In someimplementations, the sensor data includes images and/or video showing atleast a portion of what robot 200 saw when it detected an error and/orassist event.

Icon 616 can display the location of the obstacle. For example, icon 616can be positioned in panel 604 relative to robot icon 608 to beindicative at least in part of the relative location, orientation,position, and/or pose of an obstacle relative to robot 200. In someimplementations, there can be an indicator depicting where robot 200detected the obstacle. For example, indicator 606 is positioned relativeto icon 616 to indicate at least in part where robot 200 detected anobstacle. In some cases, indicator 606 can include a field of viewshowing the field of view of robot 200 when it detected an obstacle,such as the obstacle indicated by icon 608.

In some implementations, display 600 can also illustrate the process forassistance. By way of illustration, FIG. 6B is a display view indicatingthat robot 200 is off path and there is an obstacle in the path of robot200 in accordance to some implementations of the present disclosure. Indisplay 600, path 630 indicates the path which robot 200 (shown as roboticon 608) is configured to travel. Icon 626 illustrates the relativelocation on the path of robot 200. Icon 628 illustrates the location towhich robot 200 should proceed in order to clear the obstacle. In somecases, an operator can move robot 200 to the location indicated byindicator 626, such as by driving, carrying, manipulating, and/orotherwise causing to translate. Icon 622 is a zoom icon that can allow aviewer of display 600 to zoom in and/or out in display 600, enabling theviewer the potential to better evaluate the situation.

FIG. 6C is a display view showing the continuation of robot 200 afterassistance in accordance to some implementations of the presentdisclosure. In FIG. 6C robot 200 has proceeded to the location of icon628 illustrated in FIG. 6B. Accordingly, robot 200 can proceed along thepath illustrated by path 630.

Returning to FIG. 5, Block 506 includes determining if an error and/orassist event has been addressed. Robot 200 can determine if assistancehas been received in one or more ways. In some implementations, robot200 can use one or more sensors of sensors unit 312 to determine if theerror and/or assist event has been resolved. For example, robot 200 canuse one or more of sensors unit 312 to determine if obstacles are nolonger in the way, fluids are filled, hardware and/or software has beenrepaired, and/or any fatal errors, internal errors, failure of the robotto proceed with tasks, a robot getting stuck, and/or any other eventthat can give rise to an error and/or assist event have been addressed.

In some cases, robot 200 can have a button, such as the button discussedwith reference to FIG. 6A. An operator and/or other user can push thebutton if assistance is given.

In some cases, external sources can be used to determine if an errorand/or assist event has been resolved. For example, a remote network 402can review sensor data of robot 200 and/or other data to determine ifthe error and/or assist event has been resolved. As another example,external sources can include other sensors. By way of illustration,robot 200 can be operating in view of other sensors such as cameras(e.g., security cameras, thermal cameras, closed-circuit televisioncameras, and/or other cameras), motion detectors, microphones,internet-of-things devices, and/or other sensors. These sensors canprovide additional information that might not be picked up by thesensors of robot 200. Advantageously, such external sources can bepresent in the environment in which robot 200 is operating anyways.Accordingly, the external sources may not require additionalinstallation, costs, resources, etc.

FIG. 7 is a side view of external sensor 700 providing information onrobot 200 in accordance with some implementations of the presentdisclosure. As illustrated, one external sensor, external sensor 700, isshown. However, in some implementations, a plurality of external sensorscan be positioned around robot 200.

In some implementations, to determine if an error and/or assist eventhas been resolved, the data from an external sensor can be used. In someimplementations, sensors in an environment can be pre-registered and/orpredefined. For example, robot 200 and/or a remote network 402 can havea list (e.g., stored in memory) of external sensors in an environment ofrobot 200. When robot is determining if the error and/or assist eventhas been resolved, robot 200 and/or remote network 402 can send a signalto one or more external sensors on the list. For example, the signal canbe a command (e.g., to send data), interrogatory, query, and/or othersignal to have an external sensor (e.g., external sensor 700) sendsensor data. In some cases, the list can be updated by the proximity ofrobot 200 to the external sensors. For example, robot 200 can send asignal with predetermined range, such as BLUETOOTH, radio frequency,near-field communication, and/or other forms of wireless communicationto external sensors within range of such signal. Advantageously, if theexternal sensors are in range of such signal, there is an increasedprobability that robot 200 is within view of the external sensors.

In some implementations, external sensors can recognize robot 200. Forexample, a camera can have recognition software configured to identifyrobot 200. When the external sensors detect robot 200 has stopped for anerror and/or assist event, the external sensors can push data to robot200 and/or remote network 402. To aid the recognition software, robot200 can have further indications that robot 200 has an error and/orassist event. For example, robot 200 can have a light that turns on withpredetermined characteristics (e.g., turn/off frequency, color,intensity, etc.), a screen display, heat signature, sound, and/or otherindicia that would be observable by the modality of the external sensorwhen robot 200 has an error and/or assist request. Advantageously, robot200 would have externally observable indicia of an error and/or assistevent to inform operators in the vicinity of robot 200 that an errorand/or assist event has occurred.

The external sensors can also detect the presence of errors and/orassist events by recognizing patterns in sensor data. For example, theexternal sensors can detect obstacles, spills, internal errors, fatalerrors, failure of the robot to proceed with tasks, a robot gettingstuck, and/or any other event that can give rise to an error and/orassist event in substantially similar ways as described relating tosensors unit 312 of robot 200, wherein the sensor is instead an externalsensor. By way of illustration, where an external sensor is a camera,the sensor data from the external sensor can be processed by theexternal sensor, robot 200, and/or remote network 402. Accordingly, theexternal sensor, robot 200, and/or remote network 402 can detect thepresence of errors and/or assist events by identifying predeterminedpatterns in the sensor data. By way of illustration, a thermal cameracan detect spills and determine if a spill is present in the field ofview of the external sensor. As another illustration, the camera candetect an obstacle in front of robot 200 and determine if the obstaclehas moved. As another illustration, the camera can detect the positionof robot 200 and determine if robot 200 has moved. Movement of robot 200can be indicative at least in part that robot 200 has been assistedand/or the error has been resolved.

In some implementations, when robot 200 receives sensor information thatindicates an error and/or assist event from an external sensor, it candetermine that an error and/or assist event has not been resolved. Forexample, where the external sensor is configured to push information torobot 200 and/or remote network 402 when there is an error and/or assistevent, the fact that robot 200 has received sensor information from theexternal sensor can be indicative in part that the error and/or assistevent has not been resolved.

If robot 200 determines that the error and/or assist event has beenresolved, robot 200 can continue operation, as illustrated in block 508.Continuing operation can include performing a task that robot 200 hadbeen performing and/or is supposed to perform. Continuing operation canalso normal operation of robot 200. In some cases, robot 200 can receiveadditional maintenance, repair, and/or assistance to more fully resolvean error and/or assist event. As a result, continued operation caninclude continued operation of robot 200 with impaired functionality. Insome cases, robot 200 can be operated to receive additional maintenance,repair, and/or assistance, such as by causing robot 200 to move to arepair center. In some cases, robot 200 can perform an initializationbefore continuing operation. For example, robot 200 can be moved by anoperator and can re-localization using mapping and localization unit324. Robot 200 can also recalibrate its sensors and/or perform a statuscheck in order to determine the status of one or more components and/orthe entirety of robot 200 before proceeding with operations.

Where in block 506, robot 200 determines that an error and/or assistevent has not been resolved, block 510 can be performed. Block 510 caninclude attempting to address the error and/or assist event anddetermining if the error and/or assist event has been addressed. Theattempt to address the error and/or assist event can occur after apredetermined delay relative to block 506 and/or block 504.Advantageously, such a delay can give time for the error and/or assistevent to be addressed (e.g., by an operator and/or remote network 402).In some cases, such a delay can allow resources of robot 200 to not beexpended if the error and/or assist event is already being addressed.Moreover, attempting to address the error and/or assist event can, insome cases, interfere with attempts by operators to address the errorand/or assist event.

In some implementations, the predetermined delay can be determined basedat least in part on one or more of: how much time it takes to address anerror and/or assist event, empirical evidence of how long it takes foran operator to receive an alert, changes in circumstances as detected bysensors unit 312, changes in circumstances as detected by externalsensors, consequences of down time (e.g., loss of user/operatorconfidence), and/or other factors. In some cases, the predetermineddelay can be as short as a few seconds, such as 1, 2, 3, 4, 5, 6, 7, 8,9, 10 or more seconds. In some cases, the predetermined delay can belonger, such as in the order of minutes or hours.

In some implementations, block 510 can include displaying display 600for user input. In some implementations, display 600 in FIG. 6A-6C canbe dynamic. For example, display 600 can be an assist view where anoperator of robot 200 can send commands to robot 200. For example, theoperator can select the path robot 200 should take. As anotherillustration, a user can select path 610 on display 600 to tell robot200 to return and/or proceed on path 610. In some implementations, auser can illustrate to robot 200 where to go, such as by inputting a newpath (e.g., drawing on a touch screen and/or any other way of input intodisplay 600). As another illustration, a user can press a button tellingrobot 200 to proceed.

In some cases, the availability of a plurality of paths to travel canlead to an error and/or assist event. For example, robot 200 can takemultiple passes through a substantially similar area, creating aplurality of paths. In some cases, robot 200 can have difficultydetermining which of the paths robot 200 should take. An operator canthen select the appropriate path from the plurality of paths presented.

In some implementations, the plurality of paths can appear on display600, such as a plurality of paths substantially similar to path 610. Anoperator can then select the desired path for robot 200 to take. In somecases, instead of paths, positions, locations, poses, orientations, etc.can be displayed on display 600 for selection. In some implementations,an operator can input a path, such as by drawing in a path with a touchscreen and/or other input mechanism (e.g., of user interface unit 318).

In some implementations, the paths displayed on display 600 can bedetermined by through training and/or user input. For example, anoperator can manually input alternative routes to any given path,location, pose, position, orientation, etc. In some implementations,such alternatives can be generated automatically, such as by determiningone or more paths, locations, poses, positions, orientations, etc. thatrobot 200 could travel, and making those selectable. In some cases, ofthe alternatives, robot 200 can randomly select a predetermined numberto display. In some cases, robot 200 can select which alternatives todisplay based at least in part on the frequency a path is travelled(e.g., for general navigation, it might be assumed a more travelled pathis more likely to be picked; however, in some applications, such asfloor cleaning, where untraveled paths are desired, a less travelledpath can be picked), a predetermined preference, random generation,statistical probabilities, and/or any other method.

In some implementations, block 510 can include a recovery by robot 200,wherein robot 200 attempts to resolve the issue. For example, robot 200can actuate one or more actuators of actuators unit 320 in order toaddress the error and/or assist event. By way of illustration, in somecases, an error and/or assist event can be the result of noise in one ormore sensors. In some cases, the noise can have the appearance of anobstacle and cause robot 200 to mistakenly believe the obstacle ispreventing it from performing a task. Robot 200 can then make a smallmovement, such as turning, moving forward, moving backwards, and/orother movements. Such movements can, in some cases, allow robot 200 todetermine that the detection of the obstacle was due to noise andresolve the error and/or assist event (and/or consider the error and/orassist event addressed). In some cases, robot 200 can reboot, calibrate,re-localization, and/or perform processes to reset errors in theworkings of robot 200.

If the error and/or assist event (and/or a user is able to address anerror and/or assist event) has been addressed, then robot 200 cancontinue operation in block 508. Advantageously, where robot 200 canaddress the error and/or assist event, robot 200 can continue operationwith little to no operator input, potentially creating a better userexperience.

If the error and/or assist event has not been addressed, robot 200 cangenerate an alert about the error and/or assist event in block 504.Advantageously, the alert can remind the operator/user that assistanceis needed. In some applications, a predetermined delay can beimplemented. Advantageously, the predetermined alert can allow time forthe error and/or assist event to be addressed. In some implementations,the predetermined delay can be determined by one or more: the time ittakes for an error and/or assist event to resolve (e.g., the processrobot 200 would take and/or a user would take), empirical data onaddressing error and/or assist events, the speed in which theenvironment of robot 200 changes, safety concerns, and other factors. Byway of example, in some implementations, the delay can be 30, 60, 90,120, or more seconds.

FIG. 8 is a functional block diagram illustrating an assist process inaccordance with some implementations of the present disclosure. Block802 includes obstacle detection. In some implementations, obstacledetection can be performed by one or more sensors of sensors unit 312.By way of illustration, a sensor can detect an obstruction in the pathof robot 200 as it is travelling. The obstruction can appear as a soliditem along the trajectory of robot 200. In some implementations, usingone or more sensors of sensors unit 312, robot 200 can identify theobstacle through block 820, which includes object recognition. Objectrecognition can include machine learning to associate data from one ormore sensors of sensors unit 312 with identifications of objects.

After predetermined time delay 810, robot 200 can issue an alert aboutthe obstacle in accordance with block 804. The predetermined time delay810 can be substantially similar to the window of time described withreference to block 502. Advantageously, predetermined time delay 810 canallow the obstacle to move away and/or resolve itself before robot 200issues an alert about it.

The alert issued in block 804 can include an alert to remote network814, via SMS 816, and/or in assist view 818. Assist view 818 can includesystems and methods for displaying the alert and/or receiving commands,such as display 600.

After predetermined time delay 808, robot 200 can attempt to recover inblock 806. Predetermined time delay 808 can be substantially similar tothe predetermined delay discussed with reference to block 506.Advantageously, predetermined time delay 808 can also allow time for theobstacle to clear and/or robot 200 to be otherwise assisted. Recoverycan include operator assistance or actions taken by robot 200, such aswith actions substantially similar to those described with reference toblock 510.

If robot 200 is unable to recover in block 806, the process can returnto block 802 again after time delay 812. Advantageously, time delay 812can give time for robot 200 to recover in block 806. In someimplementations, the time delay 812 can be determined by one or more:the time it takes for an error and/or assist event to resolve (e.g., theprocess robot 200 would take and/or a user would take), empirical dataon addressing error and/or assist events, the speed in which theenvironment of robot 200 changes, safety concerns, and other factors. Byway of example, in some implementations, the delay can be 30, 60, 90,120, or more seconds.

FIG. 9 is a process flow diagram of an exemplary method 900 foroperation of a robot in accordance with some implementations of thisdisclosure. Block 902 includes causing the robot to perform a task in anenvironment. Block 904 includes generating data about the environmentwith one or more sensors of the robot as the robot performs the task.Block 906 includes determining from the data that the robot is at leastpartially obstructed from performing the task. Block 908 includesdisplaying a first alert indicating at least in part that the robotcannot continue the task after a predetermined first delay after thedetermination that the robot is at least partially obstructed. Block 910includes attempting to cause the robot to perform the task after apredetermined second delay after displaying the alert. Block 912includes displaying a second alert indicating at least in part that therobot is at least partially obstructed after a predetermined third delayafter the attempting to cause the robot to perform the task.

FIG. 10 is a process flow diagram of an exemplary method 1000 foroperation of a robot in accordance with some implementations of thisdisclosure. Block 1002 includes causing one or more actuators of a robotto perform a first robotic task. Block 1004 includes detecting, based atleast in part on sensor data, an obstacle while performing the firstrobotic task. Block 1006 includes actuating the one or more actuators sothat the robot is not performing the first robotic task in response to,at least in part, the detected obstacle. Block 1008 includes displayingon the user interface an assist view comprising an alert indicative atleast in part of the detected obstacle and one or more selectablepositions of the robot. Block 1010 includes receiving through the userinterface a selected one of the one or more selectable positions of therobot. Block 1012 includes actuating the actuators based at least inpart on a selected one of the one or more selectable positions of therobot, and continue performing a task substantially similar to the firstrobotic task.

FIG. 11 is a process flow diagram of an exemplary method 1100 forassisting a robot in accordance with some implementations of thisdisclosure. Block 1102 includes displaying an assist view graphicallyrepresenting that the floor cleaning robot is obstructed from cleaning.Block 1104 includes receiving operating instructions through the assistview, the operating instructions comprising displaying a plurality ofpaths of travel for the robot on a user interface, and receiving aselection of the displayed paths. Block 1106 includes causing the floorcleaning robot to clean in accordance to the operating instructions.

As used herein, computer and/or computing device can include, but arenot limited to, personal computers (“PCs”) and minicomputers, whetherdesktop, laptop, or otherwise, mainframe computers, workstations,servers, personal digital assistants (“PDAs”), handheld computers,embedded computers, programmable logic devices, personal communicators,tablet computers, mobile devices, portable navigation aids, J2MEequipped devices, cellular telephones, smart phones, personal integratedcommunication or entertainment devices, and/or any other device capableof executing a set of instructions and processing an incoming datasignal.

As used herein, computer program and/or software can include anysequence or human or machine cognizable steps which perform a function.Such computer program and/or software can be rendered in any programminglanguage or environment including, for example, C/C++, C#, Fortran,COBOL, MATLAB™, PASCAL, GO, RUST, SCALA, Python, assembly language,markup languages (e.g., HTML, SGML, XML, VoXML), and the like, as wellas object-oriented environments such as the Common Object Request BrokerArchitecture (“CORBA”), JAVA™ (including J2ME, Java Beans, etc.), BinaryRuntime Environment (e.g., “BREW”), and the like.

As used herein, connection, link, and/or wireless can include a causallink between any two or more entities (whether physical orlogical/virtual), which enables information exchange between theentities.

It will be recognized that while certain aspects of the disclosure aredescribed in terms of a specific sequence of steps of a method, thesedescriptions are only illustrative of the broader methods of thedisclosure, and can be modified as required by the particularapplication. Certain steps can be rendered unnecessary or optional undercertain circumstances. Additionally, certain steps or functionality canbe added to the disclosed implementations, or the order of performanceof two or more steps permuted. All such variations are considered to beencompassed within the disclosure disclosed and claimed herein.

While the above detailed description has shown, described, and pointedout novel features of the disclosure as applied to variousimplementations, it will be understood that various omissions,substitutions, and changes in the form and details of the device orprocess illustrated can be made by those skilled in the art withoutdeparting from the disclosure. The foregoing description is of the bestmode presently contemplated of carrying out the disclosure. Thisdescription is in no way meant to be limiting, but rather should betaken as illustrative of the general principles of the disclosure. Thescope of the disclosure should be determined with reference to theclaims.

While the disclosure has been illustrated and described in detail in thedrawings and foregoing description, such illustration and descriptionare to be considered illustrative or exemplary and not restrictive. Thedisclosure is not limited to the disclosed embodiments. Variations tothe disclosed embodiments and/or implementations can be understood andeffected by those skilled in the art in practicing the claimeddisclosure, from a study of the drawings, the disclosure and theappended claims.

It should be noted that the use of particular terminology whendescribing certain features or aspects of the disclosure should not betaken to imply that the terminology is being re-defined herein to berestricted to include any specific characteristics of the features oraspects of the disclosure with which that terminology is associated.Terms and phrases used in this application, and variations thereof,especially in the appended claims, unless otherwise expressly stated,should be construed as open ended as opposed to limiting. As examples ofthe foregoing, the term “including” should be read to mean “including,without limitation,” “including but not limited to,” or the like; theterm “comprising” as used herein is synonymous with “including,”“containing,” or “characterized by,” and is inclusive or open-ended anddoes not exclude additional, unrecited elements or method steps; theterm “having” should be interpreted as “having at least;” the term “suchas” should be interpreted as “such as, without limitation;” the term‘includes” should be interpreted as “includes but is not limited to;”the term “example” is used to provide exemplary instances of the item indiscussion, not an exhaustive or limiting list thereof, and should beinterpreted as “example, but without limitation;” adjectives such as“known,” “normal,” “standard,” and terms of similar meaning should notbe construed as limiting the item described to a given time period or toan item available as of a given time, but instead should be read toencompass known, normal, or standard technologies that can be availableor known now or at any time in the future; and use of terms like“preferably,” “preferred,” “desired,” or “desirable,” and words ofsimilar meaning should not be understood as implying that certainfeatures are critical, essential, or even important to the structure orfunction of the present disclosure, but instead as merely intended tohighlight alternative or additional features that may or may not beutilized in a particular embodiment. Likewise, a group of items linkedwith the conjunction “and” should not be read as requiring that each andevery one of those items be present in the grouping, but rather shouldbe read as “and/or” unless expressly stated otherwise. Similarly, agroup of items linked with the conjunction “or” should not be read asrequiring mutual exclusivity among that group, but rather should be readas “and/or” unless expressly stated otherwise. The terms “about” or“approximate” and the like are synonymous and are used to indicate thatthe value modified by the term has an understood range associated withit, where the range can be ±20%, ±15%, ±10%, ±5%, or ±1%. The term“substantially” is used to indicate that a result (e.g., measurementvalue) is close to a targeted value, where close can mean, for example,the result is within 80% of the value, within 90% of the value, within95% of the value, or within 99% of the value. Also, as used herein“defined” or “determined” can include “predefined” or “predetermined”and/or otherwise determined values, conditions, thresholds,measurements, and the like.

What is claimed is:
 1. A method for operating a robot, comprising:causing the robot to perform a task in an environment; generating dataabout the environment with one or more sensors of the robot as the robotperforms the task; determining from the data that the robot is at leastpartially obstructed from performing the task; displaying a first alertindicating at least in part that the robot cannot continue the taskafter a predetermined first delay after the determination that the robotis at least partially obstructed; attempting to cause the robot toperform the task after a predetermined second delay after displaying thealert; and displaying a second alert indicating at least in part thatthe robot is at least partially obstructed after a predetermined thirddelay after the attempting to cause the robot to perform the task. 2.The method of claim 1, wherein the task comprises cleaning a floor. 3.The method of claim 1, further comprising receiving operatinginstructions from an operator and continuing the performance of the taskbased at least in part on the operating instructions.
 4. The method ofclaim 3, wherein the receiving of the operating instructions from theoperator comprises receiving a determination that a button of the robothas been pushed, wherein the pushing of the button is an instruction toproceed performing the task notwithstanding the obstacle.
 5. The methodof claim 3, wherein the receiving of the operating instructionscomprises receiving one or more paths of travel and displaying the oneor more paths of travel, and receiving a selection of the displayedpaths.
 6. The method of claim 1, further comprising receiving data froman external sensor, and based at least in part on the received data fromthe external sensor, determining if the robot is still at leastpartially obstructed from performing the task.
 7. The method of claim 1,further comprising identifying that the robot is at least partiallyobstructed through object recognition.
 8. A robot comprising: one ormore sensors configured to generate sensor data about an environment;one or more actuators configured to perform a first robotic task; a userinterface configured to display and receive information; and a processorconfigured to: cause the one or more actuators to perform the firstrobotic task; detect, based at least in part on the generated sensordata, an obstacle while performing the first robotic task; actuate theone or more actuators so that the robot is not performing the firstrobotic task in response to, at least in part, the detected obstacle;display on the user interface an assist view comprising an alertindicative at least in part of the detected obstacle and one or moreselectable positions of the robot; receive through the user interface aselected one of the one or more selectable positions of the robot; andactuate the actuators based at least in part on a selected one of theone or more selectable positions of the robot, and continue performing atask substantially similar to the first robotic task.
 9. The robot ofclaim 8, wherein the robot further comprises a communications unitcommunicatively coupled to a remote network configured to monitor therobot.
 10. The robot of claim 8, wherein the processor is furtherconfigured to identify the obstacle.
 11. The robot of claim 10, whereinthe processor is further configured to learn to identify the obstaclethrough a machine learning process.
 12. The robot of claim 8, whereinthe robot comprises a floor cleaner.
 13. The robot of claim 8, whereinthe first robotic task comprises navigation of a predetermined route.14. The robot of claim 8, wherein the selectable positions of the robotcomprise differing trajectories for the robot.
 15. A method forassisting a floor cleaning robot comprising: displaying an assist viewgraphically representing that the floor cleaning robot is obstructedfrom cleaning; receiving operating instructions through the assist view,the operating instructions comprising displaying a plurality of paths oftravel for the robot on a user interface, and receiving a selection ofthe displayed paths; and causing the floor cleaning robot to clean inaccordance with the received operating instructions.
 16. The method ofclaim 15, wherein displaying the assist view further comprises:receiving data from a sensor of the robot; identifying an assist eventbased at least in part on the received data; and generating a graphicindicating the identified assist event.
 17. The method of claim 15,further comprising transmitting the assist view to a remote network. 18.The method of claim 15, wherein the causing of the floor cleaning robotto clean further comprises: causing the floor cleaning robot to navigatealong a predetermined path; and causing the floor cleaning robot toscrub the floor along the predetermined path.
 19. The method of claim15, further comprising delaying the causing of the floor cleaning robotto clean in accordance with the received operating instructions at leasta predetermined time from the displaying of the assist view.
 20. Themethod of claim 15, wherein the causing of the floor cleaning robot toclean in accordance with the received operating instructions occursafter receiving a signal indicating that an assist button on the robothas been pushed.